Hello everybody and welcome back to the session about visualization of NAM payer race and regression
so Islamization of Man by race works actually in the very same way as its lists.
So this should be a repetition of we already know about line plots scatter plots and his style grams
and we will also have a look at the regression analysis.
And it works very easy with NUM pi and not only perform linear but also polynomial regression.
And last but not least to try to approximate an assumed functional relationship.
Now given set of data points with the Taylor serious expansion techniques and how does definitely more
advanced topic in this course but yeah we will see it later in detail and hopefully understand how it
works.
All right.
So let's first of all important campaign.
And then we implement plot let the PI plot as pure here and we have to submit plot live in Lanyon Jupiter
notebook and we use them the style seaborne and then we create a normally distributed random numbers
10000 numbers with mean 5 and standard deviation to so we get an array y with 10000 elements and then
we make an histogram so we pass our NDA array right to the histogram we set binds to a hundred and a
we label the data data then we enlarge the size of the figure with purity figure.
Fixed size equals 10 6 and reset to the title of our histogram to frequency distribution of Y and we
call our X label y and our y label frequency and also we enable a legend in our histogram.
So let's let's run the circle and see what we get here yeah.
So we get here and histogram with normally disputed numbers and already look like a bell shaped curve.
So we have 10000 data points and our if you further increase this to a hundred thousand or 1 million
that what would be a very good bell shaped curve and we can also hear plot a vertical line into our
graph in the vertical line is that the value of the mean of our ending array.
So this gives us a quite good impression.
So we determined and I want a random process and that the means would be 5 and the mean of our random
numbers is actually 5 roundabout.
All right.
So and there's a function called NPL Lynn space and what MP Lynn space does that creates evenly spaced
numbers over a specified interval so we specify yeah let's press shift tab you specify here our stop
point that's 1 Our stop point that's 10 and then we say we want to have 50 evenly spaced numbers between
1 and 10.
So let's let's see what we get here.
So by default the number is 50 but we set it here to 10 so we get 10 evenly spaced numbers in the interval
between 1 and 10.
And yeah we get actually one two three four five six seven eight nine ten.
So let's create 1000 evenly spaced numbers in the interval minus 10 to 10 and call it x.
So now we have an umpire array X with 1000 numbers.
And this is very helpful when we want to plot a graph or a function.
So with Lynn space we can create many x values that are close together and then we can calculate a functional
relationship over our x values and then we can calculate y values depending on our x values and the
functional relationship so let's assume here we have our functional relationship.
So why is this three times x to the power of three minus two times X to the power of two minus five
X minus five.
So now this is we already know a vector rights operations.
So for each element of our end the array x you get a corresponding y value here.
Here we have the array y.
So now we have x and y values and they are we can actually plot these in a two dimensional graph so
we plot x and y
and they can see here our function which we are determined here and we can also define another function.
So for example the symbols function.
So why is the the symbols of X
so now I fear repeating the plotting So now why stands for the symbols of X.
And yet we get the symbols plot.
All right so far we had a histogram and a plot.
Let's go on with the scatter plot.
We generate 20 normal distributed numbers with the mean 10 and standard deviation 2 and reshape those
into two rows of 10 columns.
That's called The Matrix m so we have two rows one row two rows with each.
10 elements.
And then we slice them our matrix and we and we call our first row a.
And the second row B.
Now we have two arrays with the 10 elements each.
And then we can make us get a plot of a and b
OK so let's see here.
So our first point is.
So ace on the x axis and peace on the y axis.
And for example at C so we have our first point it's on the XXL seven point eight and then the Y eight
point six so seven point eight and eight point six.
So this is uh this point here and maybe another point.
So we have to point this on the x axis twelve and on the y axis nine point eight
and this would be this point here.
And what we can do now we can do a linear regression that fits a straight line into our graph here and
we can do this with the Frankston MP poorly fit.
So we have to pass our two arrays A and B and we have to determine the degree of our polynomial.
So one stands for a linear two A for grade quadratic function and 3 stands for cubic.
And so on but we will see this later.
So now let's run here the settle and what end people if it returns actually it returns the polynomial
coefficients from the highest power to the first and a tuple so we have here minus 0.01 5 and Twelfth
and this gives us actually the functional relationship of our linear regression.
So the functional relationship is here actually p equals ten point five 1 minus 0 point 1 5 9 9 times
a.
So this is here the intercept of our function and this is the first coefficient.
All right so now let's create some X and Y value so we create X well you to settle in space and we can
create our y values with the function and P poorly are so empty Pali y creates the y values given x
values and the polynomial coefficients rect one which we calculated here with and P poorly fit so let's
calculate both x and y values and then now we can plot our linear regression line so we plotted in our
graph where we also have our shadows here of a points that we plot in our linear regression line with
our x and y values so let's execute here
the Cicely linear regression line so a progressive line is the line that best fits to the data.
What does it mean.
So in very simple words.
So each point has a distance here from its actual y value to the to the regression line and what the
progression line does is actually minimizes to say the sum of all distances so detailed it minimizes
the sum of all squared distances.
So an easy word is just them.
The line that best fits here into into our data.
All right.
So what we can do now we can make also a crude rhetoric regression by setting the degree to two.
So let's run it here and then we get here the polynomial coefficients sixteen point nine eighths of
the intercept 0 point 0 5 is that the coefficient it's the highest power and the functional relationships
looks like this here and we can also calculate the cubic regression so with the polynomial degree 3
and this gives us a functional relationship between a and b in this manner here.
So yes here the polynomial coefficients and then we can also plot in our cubic and quadratic regression
on our graph thoughts already here.
And let's plot both so and you can see that this is the graph for the quadratic regression and the screen
graph here is the graph for the cubic regression and the more you increase the degree of the regression
the better the graph fits our data points.
So the average absolute distance between our cubic graph and the data points it's much lower than the
average distance between the data points and our linear regression graph.
And last but not least we can have a perfect regression where all the data points are on our regression
graph.
So the distance between our data points and the graph is zero.
And of course therefore minimized and in our case we have 10 data points and we can reach a perfect
regression with a polynomial degree of ten minus one.
So with some degree of nine so let's set here perfect regression and we set end p poorly fit we pass
our arrays a and b and we say okay our degrees should be the length of a and b of course minus 1.
So now we get to a 10 polynomial coefficients and now we can plot our data points and our perfect regression
curve and they can see the regression curve perfectly fits our data points and you can see here.
So we have a different scale than here with Alice get us.
So it's here 9 to 14 on the y axis.
And here it's the zero to 800.
So therefore let's assume in
and there you can even better see that our regression curve perfectly fits our data points here.
So what is a perfect regression useful for us on our case to we created to erase a and b that are not
actually independent and randomly created.
So that's definitely a no functional relationship.
And not only our perfect regression graph all of our regression curves do over fit the data so they
indicate a relationship where there's actually none.
And where fitting is the real problem in data science.
But I mean on the other hand side to feel sure that there's a functional relationship and we do not
have the math skills or tools to actually express or to find the functional relationship then it might
be a good idea to approximate dysfunctional relationship with the or the Taylor series expansion so
what we did here is actually a Taylor serious expansion.
All right so now we are finished with the session and I hope you enjoyed it.
And yes your next session.